This another example of method of thinking I dislike—thinking by very loaded analogies, and implicit framing in terms of zero sum problem. We are stuck on a mud ball with severe resource competition. We are very biased to see everything as zero or negative sum game by default. One could easily imagine example where we expand slower than AI, and so our demands always are less than it’s charity which is set at constant percentage point. Someone else winning doesn’t imply you are losing.
There are two main challenges: complexity of human values and safe self-modification. In order to correctly define the “charity percentage” so that what the AI leaves us is actually desirable, you need to be able to define human values about as well as a full FAI. Self-modification safety is needed so that it doesn’t just change the charity value to 0 (which with a sufficiently general optimizer can’t be prevented by simple measures like just “hard-coding” it), or otherwise screw up its own (explicit or implicit) utility function.
If you are capable of doing all that, you may as well make a proper FAI.
This another example of method of thinking I dislike—thinking by very loaded analogies, and implicit framing in terms of zero sum problem. We are stuck on a mud ball with severe resource competition. We are very biased to see everything as zero or negative sum game by default. One could easily imagine example where we expand slower than AI, and so our demands always are less than it’s charity which is set at constant percentage point. Someone else winning doesn’t imply you are losing.
What you describe is arguably already a (mediocre) FAI, with all the attendant challenges.
With all of them? How so?
There are two main challenges: complexity of human values and safe self-modification. In order to correctly define the “charity percentage” so that what the AI leaves us is actually desirable, you need to be able to define human values about as well as a full FAI. Self-modification safety is needed so that it doesn’t just change the charity value to 0 (which with a sufficiently general optimizer can’t be prevented by simple measures like just “hard-coding” it), or otherwise screw up its own (explicit or implicit) utility function.
If you are capable of doing all that, you may as well make a proper FAI.